Computing the Kullback-Leibler Divergence between two Generalized Gamma Distributions

We derive a closed form solution for the Kullback-Leibler divergence between two generalized gamma distributions. These notes are meant as a reference and provide a guided tour towards a result of practical interest that is rarely explicated in the literature. 1 The Generalized Gamma Distribution The origins of the generalized gamma distribution can be traced back to work of Amoroso in 1925 [1,2]. Here, we are concerned with the three-parameter version that was later introduced by Stacy [3]. Its probability density function is defined for x ∈ [0,∞) and given by f(x | a, d, p) = p ad xd−1 Γ (d/p) exp [ − (x a )p] (1) where Γ (·) is the gamma function, a > 0 determines scale and d > 0 and p > 0 are two shape parameters. We note that, depending on its parametrization, this unimodal density may be skewed to the left or to the right. Moreover, the generalized gamma contains other distributions as special cases. For d = p, it coincides with the Weibull distribution, and, if p = 1, it becomes the gamma distribution. Setting d = p = 1 yields the exponential distribution, and, for a = 2, p = 1, and d = k/2 where k ∈ N, we obtain the χ distribution with k degrees of freedom. As a flexible skewed distribution, the generalized gamma is frequently used for life-time analysis and reliability testing. In addition, it models fading phenomena in wireless communication, has been applied in automatic image retrieval and analysis [4,5,6], was used to evaluate dimensionality reduction techniques [7], and also appears to be connected to diffusion processes in (social) networks [8,9,10]. Accordingly, methods for measuring (dis)similarities between generalized gamma distributions are of practical interest in data science because they facilitate model selection and statistical inference. 2 The Kullback-Leibler Divergence The Kullback-Leibler divergence (KL divergence) provides an asymmetric measure of the similarity of two probability distributions P and Q [11]. For the case where the two distributions are continuous, it is defined as